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Abstract 


Under  an  age  replacement  policy  we  replace  at  failure  or  at  the  end  of  a 
specified  time  Interval,  whichever  occurs  first.  This  makes  sense  if  a  fail- 
vire  replacement  costs  more  than  a  planned  replacement  and  the  failure  rate  is 
strictly  increasing.  We  assume  that  the  failure  distribution  is  a  Weibull  dls 
trlbution  with  known  shape  parameter  (greater  than  one )  euad  unknown  scale  para 
meter.  Moreover,  we  assume  that  a  natural  conjugate  prior  distribution  with 
specified  parameters  is  at  hand  which  we  modify  after  each  stage  according  to 
Bayes'  rule.  Our  policy  adapts  to  the  changing  prior.  We  see  intuitively 
that  the  larger  the  replacement  interval  set,  the  more  information  we  are 
likely  to  obtain.  We  take  account  of  this  in  a  precisely  defined  manner  via 
dynamic  programming.  The  optimal  policy  is  partially  characterized  and  vari¬ 
ous  limiting  results  eure  obtained. 


Introduction  and  Sumaciry 


under  an  age  replacement  policy,  we  replace  at  failure  or  at  the  end 
of  a  specified  time  interval,  whichever  occurs  first.  An  age  replacement 
policy  makes  sense  when  a  failure  replacement  costs  more  than  a  planned  re¬ 
placement  and  the  failure  rate  is  striclty  increasing. 

The  case  where  the  failixre  distribution  is  known  has  been  treated  by 
several  authors;  see,  t.g.,  [l]  .  When  the  criterion  is  expected  cost  per 
unit  time  over  an  infinite  horizon,  the  optimal  replacement  interval  to  set 
is  found  as  an  elementary  application  of  renewal  theory.  the  opposite  ex¬ 
treme,  we  may  know  virtually  nothing  about  the  failure  distribution.  Moreover, 
as  we  acquire  failure  data,  we  ray  decide  not  to  use  the  information.  An  inter¬ 
mediate  case  is  where  we  have  partial  information  about  the  failure  distribution 
eind  adapt  our  replacement  interval  to  additional  information  about  the  failure 
distribution  acquired  sequentially. 

If  our  Initial  information  about  the  failure  distribution  seems  too  dif¬ 
fuse  to  quantify  sensibly,  we  must  choose  a  policy  that  seems  reasonable  to  use 
until  enough  data  is  accumulated.  How  much  is  enough  seems  to  be  a  qualitative 
Judgment.  Scheduling  no  planned  replacement  for  the  first  few  stages  is  prob¬ 
ably  the  best  resort  in  view  of  the  following  result: 

With  the  criterion  of  expected  cost  per  unit  time  over 
an  infinite  horizon,  the  mlnimax  strategy  over  all  fail¬ 
ure  distributions  with  the  same  mean  is  to  replace  only 
at  failxare. 

For  a  proof,  see  [l.  Section  U,2,l]„  An  analogous  result  for  discounting  is 
given  in  Appendix  I. 

The  major  assumption  that  ve  shall  make  is  a  strong  one:  viz,,  the  fail¬ 
ure  distribution  is  known  to  be  a  Welbull  distribution  with  given  shape  para- 


meter  greater  than  one;  i.e., 


(1) 


F,(y)  - 


{ 


0, elsewhere 


y  ^  0 


k  >  1  and  known  . 

This  distribution  has  strictly  increasing  failure  rate  Ucy  ”  . 

When  X  Is  unknown,  which  except  for  Section  I  we  assume  to  be  the  case, 
there  is  a  trade-off  between  minimizing  the  expected  loss  on  the  current  stage 
(■  period  starting  Just  after  the  last  replacement  and  ending  Just  after  the 
next  replaceisent )  and  acquiring  maximal  information  about  the  failure  dis¬ 
tribution  so  as  to  minimize  future  expected  losses.  An  in-service  failure 
Is  more  informative  than  a  planned  replacement,  because  In  the  latter  case 
sill  we  would  know  Is  that  the  life  of  the  Item  would  have  exceeded  the  replace¬ 
ment  Interval.  The  longer  the  replacement  interval  set,  the  more  likely  a 
failure  replacement.  However,  setting  am  infinite  replacement  interval  ■’s 
clearly  not  necessarily  optimal. 

We  take  account  of  these  intuitive  considerations  in  a  precisely  defined 
way  via  the  empirical  Bayes  approach  [I3]  and  dynamic  programming. 

Except  for  Section  8,  ve  shall  be  dealing  with  an  infinite  horizon.  Con¬ 
tinuous  discounting  is  used,  with  the  loss  incurred  at  the  time  of  replacement 
emd  the  total  loss  equal  to  the  sum  of  the  discounted  losses  Incarred  on  the 
1  ndivldual  stages.  Suppose  that  a  stage  starts  at  time  t  and  we  set  a  re¬ 
placement  interval  a  ,  chosen  from  the  extended  half  line  [O,  «],  If  a  re¬ 
placement  actually  occurs  at  t+x  ,  then  the  loss  incurred  on  that  stage  is 


(2) 


L(a,x,t)  ' 


Cie 


^2^ 


-a(t+a ) 
-ait+x. ) 


if  X  as  a 
if  X  <  a 


where  0  <  c^^  <  c^  and  a  is  a  positive  discount  rate.  The  cost  of  a 
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planned  (failure)  replacement  is  0^(02). 

Before  going  on  to  the  main  part  of  the  paper  (Sections  2-6),  we  consider 
in  Section  1  the  case  where  X  is  known.  In  Section  2,  we  give  the  dynamic 
programming  formulation  of  the  (adaptive)  case  where  \  is  unknown.  Various 
plausible  limiting  properties  are  phown  to  hold  in  Section  3.  In  Section  4, 
we  show  that  an  optimal  policy  exists.  We  consider  in  S-  ction  5  an  extension 
to  the  case  where  I  (>  l)  items  are  in  operation  simultaneously.  In  Section 
6,  we  partially  characterize  the  optimal  policy  emd  discuss  its  computation. 

In  Section  7  (nonnegligible  cost  of  adapting)  and  Section  8  (finite  horizon), 
the  loss  function  given  by  (2)  is  modified  and  a  laarkedly  different  formulation 
results.  Tills  is  followed  by  four  appendices. 

REIATED  PROBLEMS.  Adaptive  maintenance  policies  for  setting  optimal  inspection 
intervals  were  considered  in  [?].  The  information- curren-c  loss  trade-off  is 
Ignored,  but  taking  account  of  it  seems  to  be  much  harder  for  an  inspection 
policy  than  for  an  age  replacement  policy.  The  difficulty  is  due  to  the  ap¬ 
parently  intractable  likelihood  function  in  the  former  case. 

Consider,  however,  the  following  problem:  determination  of  the  optimal 
’bum-in"  time  (with  respect  to  an  appropriate  loss  function)  to  eliminate  in- 
service  failures  due  to  "infant  mortality".  If  the  failure  distribution  Is  a 
Welbull  distribution  with  known  shape  parameter  k  <  1  ,  our  methods  can  be 
readily  applied.  For  a  nonparametric  approach  to  this  problem  based  on  the 
assumption  of  decreasing  failure  rate,  see  [lO],  Note  that  failure  times  for 
those  items  that  survive  the  ’burn-in"  period  are  not  generally  available. 
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1.  Knovn  Scale  Parameters 

In  this  section  we  assume  that  the  scale  parameter  X.  in  (l)  is  knovn. 
Although  this  case  is  not  our  principal  concern,  the  results,  besides  having 
intrinsic  interest,  will  be  needed  later  when  we  consider  the  asymptotic  pro¬ 
perties  of  an  adaptive  policy. 

The  expected  loss  due  to  replacement  of  an  item  put  in  service  at  time 
t  is  e”^<t>(a)  ,  ^ere 

(3)  *(a)  -  ^  +  cj"  e-®‘dF^(x)  . 

From  the  fact  that 

(4)  5^^  ^-(OB  +  xa'') 


(Cg  ■ 


we  see  that  <(>(&)  looks  like  one  of  the  following: 


Figure  1.  Possibilities  for  ♦(a). 

i.e.,  it  crosres  the  line  corresponding  to  no  plann.ed  replacement  once  or 
not  at  I  11.  As  we  shall  see,  the  optimal  a  to  set  at  each  stage  lies  be¬ 
low  the  solid  portion  of  the  curve. 

It  is  convenient  to  comp’ite  the  risk  starting  at  time  0  .  If  the 

ert 

starting  time  is  t  ,  multiply  by  e  .  Since  the  horizon  is  infinite,  the 
(total)  risk  at  each  stage  is  the  same,  except  for  a  discount  factor;  more- 
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over,  the  optimal  replacement  interval  to  set  does  not  depend  on  the  starting 
time.  Due  to  this  stationarity,  there  is  some  fixed  replacement  interval 
which  is  optimal  to  set  at  each  stage. 

Let  denote  the  age  of  the  i^^  item  at  its  replacement  time. 

When  the  seune  replacement  Interval  is  set  each  time,  (X^)  is  a  sequence 
of  independent  and  identically  distributed  random  variables.  If  we  always 
choose  a  replacement  interval  a  ,  the  risk  is 


(5) 


r  ] 

R(a)  =  EL{a,X  ,0)  +  E  I  e"^^l  L(a,X^,0)  I 

A.  L  c  ^ 


+  E 


,-q(Xi+X2)  L(a,X^,0) 


=  EL(a,X^,0)  +  [Ee"^ll|EL(a,X2,0)] 

+  j^Ee"®^!]  .  J^EL(a,X^,0)]  +  •  •  . 


=  <t>(a)  +  5(a)4<(a)  +  ;o^(a)0(a)  +  .  .  . 


- 

1  -  6(a) 


where  the  expected  discount  per  stage  is 


(6)  E,(a)  =  e-*'®  *  ^  ) 

The  relation  R(a)  =  <l’(a)/[l 
optimality  equation. 


r  ^ 

+  e  dF,(x)  . 

-  6(a)]  can  also  be  obtained  directly  from  the 


The  minimum  risk  is 


(7) 


R  o  min  R(a) 
a 


R(a  )  ,  say 


Immediate  consequences  of  the  theorem  In  Appendix  II  are 


(1) 


* 

a  <  00 


(li) 

(3) 


* 

a  satisfies 


c,  +  ^(a  )/(l  -  &(a  )Ja')k-l 

- 


-2  -  Cj^)>k 


(lii)  a  is  unique.  Although  (8)  may  have  more  than  one  root,  there 
Is  exactly  one  root  of  (8)  that  minimizes  R(a)  . 


A  procedure  for  finding  successive  approximations  converging  monotonely 
up  to  a  that  does  not  involve  obtainirig  all  the  roots  of  (8)  is  given  in 
Appendix  li.  From  the  first  step  of  this  procedure  we  have 


(iv) 


* 

a  > 


CfO 


-  c 


1 


1 

k-1 


a  result  that  can  also  be  obtained  from  inspection  of  Fig.  1,  the  fact  that 
1  -  6(a)  is  Increasing,  and  a  simple  dominance  argument. 
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2.  Unknown  Scale  Parameters 


Whereas  the  scaJLe  parameter  was  assumed  known  in  Sc-ction  1,  we  shall 

assume  from  now  on  thatit  has  a  fixed  (but  unknown)  value  }P  .  Moreover, 

we  shall  assume  that  we  have  at  hand  a  prior  distribution  G  with  specified 

parameters  which  we  modify  after  each  steige  according  to  Bayes '  rule.  If 

I 

G  has  density  g  ,  then  the  posterior  density  =  g  x  likelihood  function  x 

normalizing  factor  independent  of  k  .  The  likelihood  itinction  in  case  of 

"ka^  k  *1 

planned  rfplacement  at  a  [failure  replacement  at  x]  is  e  [k^x  e 
Taking 


] 


(9) 


g(X;h,c) 


bC^c-le-bk^T(c)  , 

0,  elsewhere 


the  posterior  density  is  again  a  g;.mma  density 
k  k 

g(7v;  b+a  ,  c)  [g(?v;  b+x  ,  c+l)]  .  Thus,  ve  see  that  we  have  a  natural  con¬ 
jugate  prior  distribution  [12]  for  >  .  This  makes  a  dynamic  programming 
formulation  possible  with  state  variables  (b,c,t)  ,  where  (b,c)  are  the  para 
meters  of  the  prior  distribution  for  a  stage  that  begins  at  time  t  , 


2.1  Preliminaries 


Suppose  that  a  replacement  interval  a  is  set  and  replacement  occurs 
at  t+x  .  Now  we  apply  Bayes'  rule:  to  obtain  the  new  state  variables,  make 
the  transformation 


c,  t+a) 


if  X  =  a 


t 

(lO)  (b,c,t) - ^  -J 

I  (b+x  ,  c+l,  t+x)  ,  if  X  <  a  . 


Note  that  c  increases  in  steps  of  0  or  1  ,  while  b  and  t  are  re¬ 
stricted  to  half  lines.  Moreover,  although  the  risk  depends  on  t  ,  the  op- 
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timal  policy  does  not.  This  means  that  the  risk  has  to  be  computed  only  at 
points  of  a  grid  in  ;  jther  values  are  obtained  by  interpolation.. 

The  reason  for  using  a  natural  conjugate  prior  distribution  is  to  keep 
the  state  space  tractable,  a  point  which  was  stressed  in  [5].  The  re¬ 
striction  to  the  case  where  k  is  known  is  due  to  the  apparent  nonexistence 
of  a  natural  conjugate  prior  distribution  when  k  is  unknown.  Practical  ap¬ 
plication  of  our  results  to  cases  where  k  is  not  precisely  known  (depends  on 
the  sensitivity  of  the  policy  to  the  assumed  value  of  k  . 

The  expected  loss  from  the  next  replacement  when  the  state  is  (b,c,t) 
is  e"^^  (a,b,c),  where  (cf.  (3)  and  (9)) 


(ID 


4'(a,b,c) 


Cie 


-cia 


+  Cg  k  c  b 


a  -OCX  k-1  - 
e  x  dx 


For  use  throughout  the  sequel,  it  is  convenient  to  define  an  operator  T  such 
that  if  h(b,c)  is  an  arbitrary  function  of  (b,c),  then 

(12)  T[a,h(b,c)]  =  e'^(b+a^,c) 

^  r  a  ,  -oac  k-1 , 

+  keb  /  h(b+x^  c+l)  - - ^ — 

It  is  clear  from  (lO)  that  we  may  interpret  Tla,h(b,c)]  as  the  expected 
discounted  value  of  h(^',c'),  where  (b',G’,t')  are  the  new  state  variables 
and  a  replacement  interval  a  has  been  set. 


2. 2  Dynamic  Programming  Formulation 

Following  an  adaptive  policy  indefinitely,  the  minimal  risk  R(b,c)  must 


by  the  "principle  of  optimality"  ([U],[6])  satisfy 


(if.)  R(b,c)  =  min  (<t>(a,b,c)  +  T[a,R(b,c)])  . 

a 

No  solution  to  this  functional  equation  is  apparent.  However,  we  shall  find 
two  sequences  {R^jj(b,c))  and  {W^j^(b,c))  such  that  R^j^(b,c  )\lrR(b,c )  and 

WiN(b,c  )j'R(b,c ) ,  provided  in  each  case  that  the  replacement  Intervals  set  aj-e 

uniformly  bounded  away  from  0  .  The  proof  of  this  statement  for  the  first 
sequence  is  given  in  Section  3.2;  for  the  second,  the  proof  is  similar. 

We  first  find  R^j^(b,c)  .  Instead  of  following  an  adaptive  policy  in¬ 
definitely,  suppose  thiat  we  were  to  follow  an  adaptive  policy  for  N  stages. 
From  the  (N+l)st  stage  onward,  we  would  set  the  same  replacement  interval 
that  we  did  on  the  stage.  Call  the.  minimal  risk  at  stage  n  of  such 

a  policy  R^(b,c)  .  We  obtain  recursively 

(l^)  ♦(a,b,c) 

a 

where 


(15) 


and 


$(a,b,c) 


»(^b,c) 

l-5(a,b,c) 


(16) 


6(a,b,c )  =  e 


^  u  1  -  r  a  -ax  k-1  , 


and  for  n  <  N 


(17) 


(<^(a,b,c)  +  T[a, R^^^  ^(b, c ) ] }  . 


a 


n+U^' 
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»,(b.c)  is  found  in  a  similar  manner, 

(l8)  <t>(a,b,c) 

a 

and  for  n  <  N 

{<t'(a,b,c)  +  T[a,W^^^  j^(b,c)]}  . 

a  * 

We  may  think  of  W^(b,c)  as  the  analog  of  R^(b,c)  for  the  case  when  we 

adapt  for  N  stages  and  then  terminate,  it  is  felt  that  (R^j^(b,c))  con¬ 
verges  considerable  faster  than  {W^j^(b,c))  . 

Rnj^(b,c)  is  computed  for  a  finite  grid  2j  ,  n  =  N,...,l  .  Starting 
from  an  initial  value  of  c  ,  the  spacing  between  successive  values  of  c  is 
one  (cf.  (lO)).  For  values  of  b  off  grid,  R^(b,c)  is  obtained  by  inter- 

politl6n. .  We  may  choose  to  continue  computing  policies  for  successive  values 
of  N  until 

(20)  max  [(R^  j^_^(b,c)  -  R^j^(b,c))/Rj_  j^_^(b,c)]  <  e  , 

(b,c)GO  '  ^ 

where  €  >  0  is  a  preassigned  tolerance.  A  more  stringent  criterion  would 
be  to  continue  computing  until 

(21)  max  [(R^^(b,c)  -  W^j^(b,c )  )/R^^(b,c )]  <  € 

(b,c  )ep 

Observe  that,  if  (21 )  is  satisfied,  it  still  holds  with  W^j^(b,c)  re¬ 
placed  by  R(b,c)  .  Although  we  would  use  (21 )  with  more  confidence  than 
(20),  much  more  computation  would  be  Involved. 
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Let  a^(b,c)  be  tt  -  nlnimizlng  a  in  (17)  [in  (iL),  if  n  =  N]  . 

Always  acting  as  if  we  were  at  the  first  stage,  what  we  shall  actually  do 
/  *  \ 

at  stage  n  (n  =  1,2,...,N  ,...)  is  to  use  the  replacement  interval. 
aij^*(bn,Cn) ,  where  the  state  variables  at  stage  n  are 

N  is  the  smallest  N  satisfying  (20)  or  (21 ),  whichever  is  deemed  appro¬ 
priate. 
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Lirnltlng  Properties 

Aiialogously  to  Rj^(b,c ) [R(b,c )] ,  define  R^^j^j(b,c ) [R^(b,c ) ]  to  be  the 

mlnin'al  risk  under  the  restriction  that  the  replacement  Interval  set  at  each 
stage  be  at  least  e  ,  for  some  fixed  £  >  0  .  Let  a^(b,c )[a^(.b,c )]  de¬ 
note  the  minimizing  a  .  Our  replacement  policy  will  use  each 

stage.  We  call  this  policy  (N,  c) -optimal. 

In  applications,  restriction  to  the  class  of  (N, £) -optimal  policies  is 
no  important.  It  is  plausible  that  our  results  hold  without,  the  restriction 
that  the  replacement  intervals  set  are  uniformly  bounded  away  from  0  ,  but 
proofs  are  lacking. 

We  may  paraphrase  (roughly)  the  main  results  of  this  chapter  as  follows 

1.  After  a  sufficiently  ISLrgc  number  of  replacements,  we  are 
virtually  certain  at  any  particular  stage  to  be  setting 

a  replacement  interval  close  to  the  one  we  would  use  if 
we  knew  A  . 

2.  For  N  sufficiently  large,  )  is  "close"  to  a^(*  ;  •)  . 

We  also  consider  what  happens  for  small  discount  rate  a  . 

5. 1  Asymptotic  Optimality 

An  ( N^£)-optimal  policy  seems  to  be  a  reasonable  one  to  use  during  the 
transient  period  when  we  are  acquiring  significant  information  about  the 
failure  distribution.  Because  of  discounting,  this  is  an  important  con¬ 
sideration.  Moreover,  we  now  state  a  result  that  shows  that  an  (l!,e)- 
optimal  policy  has  an  asymptotic  optimality  property  that  we  would  require 
of  any  policy  we  were  to  use. 

THEOREM  1.  (asymptotic  optimality).  Under  an  (N,e) -optimal  policy. 
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pllm  =  V  > 

n  -►  00 

N  fixed 

* 

provided  that  a^Q  ^  e  .  In  fact,  with  ;he  same  proviso, 
plim  =  a*o  . 

To  prove  this  theorem,  we  require  some  preliminary  results.  To  save 
tedious  repetition,  we  do  not  mention  Slutsky's  theorem  [5,  Section  20,6] 
each  time  we  use  it. 


LEMMA  1.  Let  Y  have  distribution 
positive  constant.  The  variance  of 


and  X  a  mln(a,Y),  where  a  is  a 
is  maximized  at  a  ■  oo  and  is  finite. 


PROOF. 


EX^  =  ^  (1  -  ) 


Var 


,  1  -2Xa^  ^  k  ->a^ 

«k  ^  1  -  e _  2a  e 


a  ^k  ^  .2k-l.-M‘ 


"5: 


Var  X*^  =  2ka“‘  ^e  >  0 


LEMMA  2.  Under  any  policy  for  which  a^  ,  the  n^^  replacement  interveil 

set,  must  be  at  least  e  >  0  ,  n  =  1,2,.  .. , 

(i)  plim  byn  = 

(ii)  plim  c^/n  »  i  , 

where  ^  is  a  positive  constant. 
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By  Lemma 


PROOF.  Let  X  =  min  (a  ,Y),  where  Y  hae  distribution  P.n  . 

n  '  n  ’  '  ’  A'^ 

1,  the  variance  of  is  uniformly  boundei  over  n  for  all  (a^ja^,  ... 

the  bound  is  achieved  for  (oo^«o,...)  . 

Therefore,  by  a  standard  martingale  convergence  theorem  [11,  Section 

29.1], 

pllm  ~  =  pllm  ^  ) 

1 

n 

■  pii«. 

1 


n 


n 


i  Pii-  K/_, 


.0  k 
-X  ai  \ 
e  1  ) 


The  limit  is  a  positive  constant,  since  ^  ^  >  0,  Var  “*  0 

Hence,  (l)  is  proved.  The  proof  of  (li)  is  similar,  || 


LEMMA  3  .  (consistency).  Under  any  poJ.lcy  for  which  ^  e  >  0, 
P[G(X;b^,(^) converges  to  a  distribution  degenerate  at  X^]  ■  1  . 


PROOF.  The  mean  euid  variance  of  the  prior  distribution  at  steige  n  are, 

2 

respectively,  c^/b^  and  c^^/b^  .  By  Lemma  2 


c  Cj^/n 

plHnUi=  pllm 

n  n' 
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similarly, 


.0 

pllm  b^/'n  "  ^  ^ 


pllm 

»n 


pllm  ^  pllm  ^ 
“n 


-0  1 
X  pllm  ^  “  0  , 


n 


since  pllm  ■  «  by  Lemma  2.  Since  convergence  In  quadratic  mean 
convergence  in  probability -^^convergence  of  lavs,  the  lemma  follows. 


PROOF  OF  THEOREM  1,  We  observe  that  Lemma  2  Implies 


pllm  b  m  00 

n 


pllm  c  ■  « 
n 

euid  hence  it  follows  from  Inspection  of  (ll),  (l2),  and  (lU)-(l6)  that  with 

b  and  c  random  veirlables  euad  n  -*  w 
n  n 


(22) 


W.p.l 


Next,  we  note  that  (l4)  and  (15)  Imply 


(25) 


R^(b,c)  =  min  [0(a,b,c)  +  6(a,b,c)R^(b,c)] 
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Hence 


(24) 


w.p.l  by  (17)  and  (22) 


■  ®^(\>c^)  +  0(1)  v.p.l  by  (25) 

(comparing  mlnlmands) 


min 

a^€ 


♦  (a.bn.On)  ♦  6{a,b_^,c^)  } 


V.p.l 


Letting  a 

Q 

hypothesis 


staind  for  have,  by  backwards  induction  (on  the 


(25)  .  ‘^„(bn,=n)  ■ 


V.p.l;  hence 


(26) 


<N(V=n)  -  l"-  ”fS:,b„,cJ 


and  so 


(27) 


pita  I^„(b„,c^) 


■<>(a,b^,c^)  +  0(1) 

-  ^  ■ 


pllm  min 
a^€ 


We  note  tliat  by  the  Hally-Bray  theorem  [11,  Section  11.  5]  and  Lemma  3 


(26)  plim  0(a,b^,c^)  = 


(29)  plim  &(a,h^,c^)  -  bypi&) 


Since  the  minimand  in  (27)  converges  uniformly  for  a  ^  e  ,  we  may 
interchange  plim  and  min,  which  is  Justified  by  the  lemma  proved  in  Appendix 
III.  Therefore,  by  (28)  and  (29) 


(50) 


plim  I^N^^n’^n^  “ 

a^e 


1  - 


<0 . 


if 


Let 


(51) 


^^0(a) 

1-  ^xO(a)  ”  l-6(a,b  n  ) 

n'  n' 


Since  by  (28)  and  (29)  converges  in  probability  to  0  uniformly  for 

a  ^  €  ,  plim  A  (a  )  ^  plim  sup  A  (a) »  sup  plim  A  (a)  ■  0  and  hence 


(52) 


plim 


0 


Combining  (32)  with  (30)  shows  that 


(53)  plim  “  V  '  %0  ^  e  , 

proving  the  first  assertion  of  the  theorem.  Since  by  the  theorem  of  Appendix 
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II  a^Q  Is  unlquq,  the  second  assertion  follows  from  the  first.  || 

5. 2  Behavior  eis  N  -♦  <o 

Let  R(7T,b,c)  denote  the  risk  under  the  stationary  policy  tt  .  For 
example,  If  we  use  each  stage,  then  tt  ■  ^  ’)  •  Note  that 

c  e  12 

R(a  ( •  >•  ),b,c)  ■  lr(b,c).  If  we  measure  the  "distance'  from  tt  to  tt  by 

I  I  TT^  -  tt^  I  I  ■  sup  I  R(7r^,b,c)  -  R(7r^,b,c)  |  , 

(b',c)€r 

where  P  Is  a  comi)act  subset  of  ((b,c):  b  >  0,  c  >  0),  then  we  have 

THEQUEM  2.  11m  |  |  sl  , .  )-a^  (. , .  )  |  |-0 

N  • 

Two  lemmas  are  required  for  the  proof. 

LEI®<A  4.  R^(b,c)  satisfies 

(34)  Ii^(h,c)  -  min  (♦(a,b,c)  +  T[a,R^(b,c)])  ; 

a  ^  € 

moreover.  If  H(b,c)  Is  a  solution  to  (34)  such  that 

(35)  pllm  e“°^nH(b^^c^)  -  0  , 

then  H(b,c)  ■  R^(b,c)  . 

A  proof  can  be  given  along  the  lines  of  [8j. 
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LEMU  3. 

(i) 

I^jj(b,c)>l<R^(b,c)  . 

(il) 

T[.,R^(b,c)]  is  continuous  . 

(Hi) 

convergence  is  uniform  on  P  . 

(Iv) 

R^  ( •  ^  • )  is  continuous  on  P  . 

PROOF.  Since  R^j^(b,c)  la  bounded  below  by  0  and  decreasing  in  N  , 

4(b,o)  >iH(b,c),  say.  Since  Theorem  1  Implies  that  pllm 

it  follows  that  (35)  holds.  By  Lemma  k,  it  therefore  suffices  to  show  that 
H(b,c)  satisfies  (34)  to  prove  (i)  . 

Taking  the  limit  as  N  -*  «  on  both  sides  of 

r5  (b,c)  -  min  (♦(a,b,c)  +  T[a,R|jj(b,c )])  , 

a  ^  € 

we  get  H(b,c)  on  the  left.  On  the  right 


(I)  interchange  lim  and  min 

This  is  Justified  by  the  lemma  proved  in  Appendix 
III. 

(II)  take  the  limit  inside  the  integral  in  the  relation 


-ax  k-i, 

X  dx 

,  k\c+l 
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This  Is  justified  by  the  Lebesgue  monotone  convergence 


theorem. 

This  shows  that  H(b,c)  satisfies  (3^). 

It  is  easily  seen  that  (i)  _ ^(lii).  Since  the  uniform  HJtlt  of 

continuous  functions  is  continuous,  (l)  — ^(li)  and  (ill )  =z^(lv).  || 
Theorem  2  is  an  immediate  consequence  of  part  (ill)  of  Lemma  5.  Note  that 
the  convergence  of  IKn(-  J  -)  -  a^  I  I  to  0  is  not  necessarily  monotone. 

Provided  that  the  unconstrained  minimizers  are  ^  e  ,  we  conjecture 
that,,  starting  with  N  ■  2,  a^j^(b,c )  a^  (b,c  )  but  that  a^^(b,c)  ^  a^jj(b,c), 

N  ■  2,3,....  The  intuitive  basis  for  these  conjectures  is  that 
(i)  the  longer  the  replacement  Interval  set,  the  more 
information  about  the  failure  distribution  we  are 
likely  to  obtain; 

(il)  the  larger  N,  the  more  stages  we  have  to  get  in¬ 
formation  and  so  aciuirlng  maximal  information  be¬ 
comes  relatively  less  Important; 

(ill)  an  exceptional  case  occurs  for  N  «  1  because  there 
we  do  not  take  account  of  information  acquisition  at 
all. 

In  contrakt  to  these  conjectures,  any  of  the  possibilities 

af.,(t  )=a5.,(b  ..LC  ) 

IN'  n'  >  n.N'  r.-j-l’'  n+l"^ 

can  hold. 

3.  3  Behavior  as  Q  -►  0  . 

To  emphasize  the  dependence  of  4>(a,b,c)  on  a  ,  we  write 

AT 

<ti(a,b,c;a)  .  From  a  well-known  Tauberian  theorem  [l4.  Chapter  5] 
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on  Laplace-Stleltjes  transoms,  it  follows  that  the  expected  cost  per  unit 
time  (with  respect  to  our  current  prior  dletribution)  when  the  same  re¬ 
placement  Interval  a  is  set  at  each  steige  is 

A* 

lim  a<>(a,b,c;a) 
a  -•  o'*’ 

Thus,  by  taking  a  small  we  could  approximate  vhat  we  would  do  if  our 
goal  were  actually  lo  minimize  expected  cost  per  uni',  time. 

For  a  discussion  of  this  point  in  a  somewhat  different  context,  see 

[6]. 

Note  that  the  (adaptive)  problem  degenerates  when  the  loss  is  un¬ 
discounted  cost  per  unit  time  since  we  could  Ignore  the  loss  in  any  finite 
transient  period  while  we  learned  about  the  failure  distribution. 
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Existence  of  ein  Optimal  Policy 

For  any  particular  (b,c),  we  have  already  seen  (cf.  Lemma  4)  that 
an  optimal  replacement  interval  to  set  exists,  viz.  a^(b,c)  .  Although 
we  do  not  have  a  means  for  obtaining  it  exactly,  by  successive  approxi¬ 
mations  we  can  do  almost  as  well  (cf.  Theorem  2).  From  a  practical  view¬ 
point,  perhaps  this  is  all  we  should  ask. 

However,  taking  a  global  outlook,  a  policy  tt  ■  ,7r2> .  • .  ) 

is  a  sequence  of  functions  such  that  tt  tells  us  what  action  to  take 

n 

as  a  function  of  lose  nothing  here  by  considering  only 

nonrandomlzed  policies.  )  In  view  of  the  functional  equation  (34),  we 
always  act  as  if  we  were  at  the  first  stage.  Hence,  our  policy  is  sta¬ 
tionary,  i.e.  ,  TT  =  (tTj^jTT^,.  . .  ).  Following  [8],  it  can  be  shown  that  an 

optimal  policy  exists  with  respect  to  the  topology  of  pointwlse  conver¬ 
gence  [9,  Chapter  7]  using  Tychonoff’s  theorem  on  the  predict  of  compact 
spaces.  Of  course,  the  optimal  policy  is  to  use  a^('»,*)  at  each  stage. 
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5.  Extension 

Until  now  we  have  been  tacitly  assuming  that  there  was  only  one  item 
in  operation  at  any  particular  tine.  However,  suppose  that  I  items  are 
in  operation  simultajieously.  Our  previous  results  can  be  applied  to  this 
case.  For  simplicity,  we  consider  only  the  case  I  ■  2  . 

Without  loss  of  generality,  we  ass'ame  that  Just  after  the  last  re¬ 
placement  item  1  had  age  A,  item  2  was  new,  and  the  state  was  (b,c,t). 
Suppose  that  we  are  using  an  "N”  stage  policy.  Let  a  ■  a^j^(b,c)  .  We 

set  replacement  intervals  raax(o,a-A)  and  a  for  items  1  emd  2,  re¬ 
spectively. 

Case  1:  Neither  item  fails  before  max(0,a-A), 

(i)  (b,c,t)  -♦  (b+a^  +  max(0,a^  -  A^,c,t-Hnax(o,a-A) )  . 

(ii)  replace  the  older  item  if  A  >  0;  otherwise  both  items. 

(iii)  in  the  former  case,  the  item  not  replaced  has  age 
max(0,a-A). 

(iv)  proceed  as  before. 

Case  2:  The  older  (newer)  item  is  the  first  to  fail;  it  fails  at 

t  +  X  . 

(i)  (b,c,t)  -*  (b+2x^-A^,c+l, t+x)  . 

(ii)  replace  the  older  (newer)  item. 

(iii)  the  item  not  replaced  has  age  x  (A+x). 

(iv)  proceed  as  before. 

Note  that  we  will  ofv.en  modify  a  replacement  interval  previously  set. 
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6.  Computing  the  Optimal  Policy 

Section  6.1  is  prelimlnairy.  In  Section  6.2  we  partially  characterize 
the  optimal  policy.  This  characterization  can  be  used  to  expedite  the  search 
I'or  minimi zers  in  (lU)  e^nd  (l?). 

In  computing  the  optimal  policy,  errors  can  arise  in  two  ways: 

(l)  in  numerical  integration  and  in  the  seeirch  for  mlnimizers, 
we  must  in  practice  confine  ourselves  to  a  finite  Interval. 

(li)  because  we  are  restricted  to  a  finite  grid  where  R^j^(b,c) 

is  computed  recursively  for  n  =  N,...,l  ,  there  will  be 
errors  due  to  interpolation  as  well  as  numerical  inte¬ 
gration. 

We  bound  the  error  corresponding  to  (i)  in  Section  6.3.  Th<^  magnitude  of 
the  error  corresponding  to  (ii)  depends  on  the  fineness  of  the  grid.  In 
any  case,  the  spacing  between  successive  values  of  c  is  one.  However, 
there  is  a  tradeoff  in  the  spacing  of  b  values  between  error  reduction  and 
computing  time.  No  doubt  it  is  efficient  to  use  unequal  spaclngs  determined 
during  the  course  of  the  computation,  but  we  shall  not  consider  the  matter 
further. 

An  outline  of  the  replacement  policy  computation  is  given  in  rlgo- 
rithmic  form  in  Appendix  V, 

6. 1  Possibilities  for  the  Graph  of  4>(a,b,c) 

In  contrast  to  the  situation  where  "h  is  known,  <l>(a,b,c)  looks 
like  one  of  the  five  possibilities  shown  in  Fig.  2.  Their  significance 
will  be  brought  out  in  Theorem  5. 

To  see  that  one  of  these  cases  must  hold,  we  note  that  the  slope  of 
<D(a,b,c),  given  by 
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(;6)  4>(a,b,c)  =  e 


-Ota  /  b  ^  \  fkc(c  -c  )a 

b+a*"  J  :  k  ^'1°, 

'  b  +  a 


is  negative  at  a  =»  O"^  and  I'or  all  sufficiently  large  a  .  The  case  that 
occurs  depends  on  the  number  of  positive  zeros  of  ^  <l>(a,b,c)  .  The  cor¬ 


respondence  is 


no.  positive  zeros 


case 


0 

1 


(1)  -  (5) 

(M 

(5) 


The  number  of  positive  zeros  is  2,  0,  1  as 


{tl) 


Q  >  max 
3  a 


kc(c2-c^)a 

c^(b+a^) 


k-I 


By  setting  the  derivative  of  the  maxlmand  equal  to  zero,  we  find  the  maxi- 

1  /k 

mtun  occurs  at  a  =  [(k-l)b]  '  and  that  (5?)  is  equivalent  to 


(38) 


b  >  [c(c2-c^)/c^a]^(k-l)^  ^ 


Since  the  derivative  of  the  maxlmand  has  exactly  one  zero, 
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C>(a,b,c)  can  cross  the  abscissa  at  most  twice;  i.e.,  t(a,b,c) 


has  at  most  two  positive  zeros. 


We  remark  that  for  a  sufficiently  small  case  (2)  must  hold. 


6. 2  Partial  Characterization  of  the  Optimal  Policy 


THEOREM  3.  For  n  =  N  : 

(i)  b  ^  [c(c2-cj^)/c^a]^(k-l)^''-  a^(b,c)-® 


(li)  Suppose  that  the  inequality  in  (i)  does  not  hold.  Let 

ag(b,c)  denote  the  smalJ.er  positive  zero  of 

4'(a,b,c).  Then 

♦(ag,b,c)  ^  <t>(«o,b,c)  a^(b,c)-oc 

(ill)  Suppose  that  neither  of  the  preceding  inequalities  holds. 

Let  aj^(b,c)  denote  the  root  of  4>(a,b,c)  =  4(»,b,c) 

that  is  larger  than  ag  .  Then  either  a^(b,c)  =  «b 
or  ag(b,c)  <  a^(b,c)  <  a^(b,c)  . 
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PRCX)F.  Using  simple  dominance  arguments,  the  theorem  follows  from  the  fact 
that  l-6(a,b,c)  is  increasing  in  a  ,  auid  Inspection  of  Figure  2  , 

Note  that  case  (4)  or  case  (5)  holds  iff  the  inequality  in  (l)  holds.  || 

We  conjecture  that 

(I)  |^T[a,R^(b,c)]  <  0  ,  n-N,...,l 

(II)  The  extension  of  Theorem  5  to  n  »  N,..,,l  holds. 

Note  that  (l)  =^(ll). 


6. 3  An  Error  Bound 


Let  0  <  e  <  min  (l,c, ), 


(39)  -  [c(c2-Cj^)/c^a]^(k-l)^"^ 


Vo  »  2l(co+l)(c+l)a"^r(k+l)]/€  , 


and  Vj  be  the  smallest  a  satisfying 


(41) 


-da  /  b  \c  ^  - 

t?'  ^ 


We  shall  bound  the  error  introduced  by  the  following 


PROCEDURE.  Select  a  number  a'  ,  depending  on  (b,c)  ,  such  that 


(42) 


a'  »  max 


(max(v^,V2)  -  h)^^^,v. 


Take  the  minlmizer  over  [0,a']  to  be  the  replacement  interval,  unless 
the  minimizer  is  a'  ;  in  the  later  case,  take  the  replacement  interval 
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Lo  be  m  ,  approximating  R^(b,c)  by 


(1+5)  $(a',b,c),  if  n  =  N 

<D(a',b,c)  +  T[a',R^^^^jj(b,c)]  ,  if  n<N. 

THEOREM  k.  The  risk  under  the  foregoing  procedure  exceeds  the  minimal 
risk  over  all  procedures  using  the  grid  ^  by  at  most 

2e(N-l)  +  c  , 

1  -  M 

where 

(4U)  Mamax(M’,M") 

(45)  M'  »  max(6(a',b,c)  :(b,c)e|:,  b^v^(c)} 

(46)  M"  a  max(6(^  (b,c),b,c):(b,c)ei,  b<v^(c)) 

and  ag(b,c)  is  the  smaller  positive  zero  of  ^  **>(a,b,c)  , 

To  prove  Theorem  4,  we  need  two  lemmas. 


LEMl.lA  6. 

If 

(47) 

b  >  max  V  ,  (c+l)a~^r(k+l)  , 

-  'i-  J 

then,  for 

n  a  N, . . . ,1 , 

(46) 

C2ca"^r(k+l)/(b+a^) 

,c)  ^ 

1-ca  ^r(k+l)/(b+a^) 

and 

(49) 

kcb=  P  R  . 

/  .  /v,  k\c+l 

^a'  (b+x  ) 
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Cg [ (c+1  )a"^r(k+l )/ (b+a '^ )  ]^ 

1-(c+1)q  '^r(k+l)/(b-fa'^) 

PROOF.  Observe  that  (U7)  implies  that  the  denominators  on  the  rli;ht-hand 
sides  of  (46)  and  (49)  are  positive. 

It  suffices  to  show  that  (48)  and  (49)  hold  when  n  ■  N  ,  since 

R^(b,c)  ^  Rjjj^(b,c)  .  By  Theorem  5(i),  (4?)  implies 

Note  that 

/oo 

e-ooc^k  1^ 

0  (b+x^^)'^-^^ 


^  (cpkcb^/b^'*’^)  1^  e~^x^”^cix 

^0 

*  c^ca”^r(k)/b  ; 

similarly 

(52)  &(«>,b,c)  ^  kca"^r(k)/b 

Relations  (90)-(52)  imply  (48).  It  can  be  easily  checked  that  (48) 
implies  (49)  . 


LEMMA  0.  If  0  <  G  <  min  (l,c^)  and 


(53) 

(54) 


cic 


-aa 


Cgkcb 


< 

f” 


g/2 

^  e/2 


} 
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then 


(^5)  |♦(a,b,c)  -  *{  1  E 

|5(a,b,c)  -  »{  ,b,c)  1^ 

The  straightforward  proof  is  omitted. 

Theorem  4  follows  from  the  preceding  Lemmas,  algebraic  manipulation, 

euid  Theorem  5. 
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V .  Cost  of  Adapting 

Suppose  that  recording  data  and  varying  replacement  intervals  entail 
a  nonnegliglble  administrative  cost  with  expected  value  ^(a,b,c)  per 
stage  until  we  stay  with  a  fixed  replacement  interval.  A  new  formulation 
is  required. 

1 1" 

Suppose  that  we  ”"'ll  stop  adapting  on  or  before  the  N  stage.  Let 
U^(b,c)  denote  the  minimal  risk  at  stage  n  .  Then 

iy()  Uj^(b,c)  =  Rj^{b,c) 

and  for  n  <  N 

where 


b9)  =  min  (^{a,b,c)  +  4)(a,b,c)  +  T[a,U^(b, c ) ])  . 

a 

The  functions  U^(b,c)  are  computed  recursively  for  n  »  N,...,l  over  a 
finite  grid  of  (b,c)  values.  For  values  off  the  grid,  interpolation  is 
used.  If  (b,c)  is  on  the  grid,  then  for  n  =  N,...,l  record 

(i)  • 

(ii)  whether  the  minimum  was  achieved  for  ^^+1^^*^^ 

U'  ^ (b,c)  . 
n+1  ^ 

(iii)  in  the  latter  case,  the  minimizing  a  . 

To  find  the  replacement  interval  to  set  a  stage  n'  ,  say,  when  the 

/  \  * 
state  is  (b,c,t}  ,  find  the  smallest  n  ^  n'  ,  say  n  ,  such  that 

U'  ^(b,c)  <  U^^^(b,c)  .  If  no  such  n  is  found,  we  use  aj^(b,c)  . 
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otherwise,  we  take  the  a  that  minimized  replacement  in¬ 

terval  to  set. 

Note  the  resemblance  to  the  structure  of  Bayes '  procedures  for  trun¬ 
cated  sequential  games  given  in  [2,  Section  9.2],  Bj^(b,c)  plays  the  role 
of  the  terminal  loss. 
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d.  Finite  Horizon 


Suppose  that  we  are  given  a  finite  time  horizon  H  .  We  modify  our 

/ 


loss  function  as  follows. 


(60)  L(a,x,t)  = 


-o(t+a) 

-o(t+x) 


,  if  X  ■  a,  t+a  <  H 

,  if  X  a,  t+x  <  H 
,  otherwise 


0  <  ^1  <  Cg  ,  0^0 


Since  we  now  have  a  '’inite  horizon,  it  is  possible  to  let  the  discount 
rate  vanish.  It  is  clear  that  we  may  assume  that  a  ^  H-t  S’^d  t  <  H  . 

If  we  stop  at  the  end  of  the  current  steige,  which  we  may  suppose 
started  at  time  t  ,  then  we  incur  a  terminal  loss  L  (x,t)  .  Our  re- 
suits  do  not  depend  on  its  form.  Observe,  however,  that  the  optimal  re¬ 
placement  interval  to  set  will  depend  in  general  on  t  as  well  as  on 
(b,c)  ,  in  contrast  to  the  infinite  horizon  case.  We  have  the  option  of 
stopping  immediately,  making  no  further  replacements  of  any  kind;  if  we 
take  this  option,  our  terminal  loss  is  L  (0,t)  .  In  general,  we  denote 

5 

the  expected  value  of  L  (x,t)  by  ^  (a,b,c,t)  . 

6  S 

We  assume  that  there  is  a  finite  upper  bound  N  on  the  number  of 
spares  available.  At  each  successive  stage  we  select  the  optimal  con¬ 
tinuation.  Then,  ^  la  [2,  Section  9.2)  ,  the  minimal  risk  with  N-n 
spares,  V^(b,c,t),  jatisfies 

(6l)  V^(b,c,t)  =  rain  [L^(0,t),  Q(b,c,t)]  , 


where 


(62) 


Q(b,c,t) 


r  _ 

min  ♦  (a,b,c,t)  +  e  ♦(a,b,c) 
8 
a 


aind  for  n  <  N 


(65) 


V  (b,c,t)  ■  mio 


where 


(6U) 


^n(^»c>t)  «  min  {e“^^^(a,b,c)  +  T' [a,V^(b,c, i] ) 
a 


and 


(65) 


T' [a,V^(b,c,t)  -  ^  V^(b+a^,c,t+a) 


b-fa 


♦  kcb'^  r 


(b«‘)‘ 


The  decisions 

<  (i)  whether  or  not  to  stop 

(ii)  if  not,  what  replacement  interval  to  set 

are  arrived  at  In  a  manner  similar  to  that  used  in  Chapter  VII. 

Reference  to  [l,  Section  4.2.4]  shows  that,  when  the  failure  dis¬ 
tribution  is  known,  a  much  simpler  formulation  is  possible. 
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Appendix  I 


Mlnlmax  Strategy  When  Discounting^ 

Lt-'t  the  class  of  distributions  be  defined  as  follows; 

fix  Q,v  (a>0,  0<V<l)  ;  a  distribution  F  belongs  to  the  class 

poo 

iff  V  »  I  e"^dF(x)  .  Let  F  be  the  exponential  distribution 

with  mean  (l-v)/crv  .  It  is  easily  checked  that  . 

THEOREM.  With  an  infinite  horizon  and  a  loss  function  L(a,x,t)  given 

oy  (2)  at  each  stage,  the  minlmax  strategy  over  -V  '  is  to  replace  only 

at  failure.  Replacement  only  at  failure  is  also  minimax  over  all  distri¬ 
ct  V 

butions  in  ’  having  nondecreasing  failure  rate. 

PROOF.  For  an  arbitrary  distribution  F  ,  denote  by  R(a,F)  the  risk  when 
a  fixed  replacement  interval  a  is  set  at  each  stage.  Using  a  technique 
found  in  [].,  Section  4.2.1],  we  have 


l-v 


max 


Fej 


ra,v 


R(oo,F)  ^  min 
a  ^  0 


max 

F€ 


R{a,F) 


^  max  min  R(a,F) 

a  ^  0 

CoV 

^  min  F(a,F^)  =  R(oc,Fq)  =  .  II 

a  2  0 
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Appendix  II 


e  Replacement  When  Discounting  and  the  Failure  Rate  Is  Strictly  In¬ 


creasing  to 


Let  F  have  failure  rate  )  ,  0  <  c^  <  c^  ,  and  O  >  0  .  Defl 


F(a)  =  1  -  F(a) 


♦  (a)  o  c  c'’^^(a)  +  c, 

1  i 


e"^dF(x) 


6(a)  =  e“°^(a)  +  /  e“°“dF(x)  . 


R(a  )  =  min  R(a) 

a  ^  0 

THEOREM.  If  q(a)  is  continuous  and  strictly  increasing  to  to  ,  then 


# 

a  < 


q(a*)  a  a[Cj^+R(a  )]/(c2-Cj^) 


(lii)  a  is  unique  . 


PROOF. 


d  nfr'l 

■ST  -  1-6 (a) 


(c2-Ci)q(a)-a(ci+R(a))  I  , 


which  is  positive  for  all  sufficiently  large  a  ,  since  q(a)t«  and  R(a) 
is  uniformly  bounded  for  a  ^  €  >0  . 
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(ii) 


R(a)  .  Such  a 


/  \  * 

By  (i),  a  is  a  finite  zero  of 

zero  existb  since  q(a)  oo  ,  R(a)  is  uniformly  bounded 
fo’"  ^  ^  €  >0,  R(0)  =  00  ,  and  q(a)  and  R(a)  are  con¬ 
tinuous. 

(iii)  (suggested  by  David  Matula). 

-X-  ^  -x- 

Let  and  a^  be  minimi zers.  By  (ii),  R(a^  )  = 

'X'  ^  'X‘  'X'  “X 

R(a2  ;  — ^q(a^  )  =  q(a^  )  =  ^2  ' 

is  strictly  increasing.  |! 


Note  that  R(a)  may  have  more  than  one  local  minimum,  but  that  only 

one  of  these  is  a  global  minimum.  If  part  (ii)  of  the  theorem  is  used  to 

» 

find  a  ,  all  local  minima  must  be  found  and  the  corresponding  risks  e- 
valuated.  We  suggest  a  possibly  useful  alternative  procedure. 

Let 


a 

r 

V  =  min  4'(a)  +  6(a)V 
nN  n+l,H 


n  <  N  . 


It  can  be  shown  that 

q"^(a(c^+V^j^)/ (c2-Cj^))t  a* 

by  proving  thai  ^  (using  the  method  of  proof  for  Lemma  5)  and 

comparison  with  part  (ii)  of  the  preceding  theorem.  The  heuristic  basis 

for  this  is  that  would  be  the  minimal  risk  if  we  were  to  terminate 

* 

after  N  stages.  A  monotone  improving  approximation  to  a  is 

q"^(a(c^+V^j^)/  (c^-c^))  , 


where  is  computed  by  recursion.  The  minimizing?  a  for  is 


q-^(a(c^+Vn^  )/(Cg-c^)) 


where  we  set  V.,  ,  „  =  0  .  Note  that  q'^  exists  since  the  feillure 
N+1 ,  N 

rate  is  continuous  and  strictly  increasing  to  »  .  For  the  Welbull  dis¬ 
tribution  defined  by  (l),  q  ^(x)  a 


-59- 


Appendix  III 


Interchange  of  llm  and  min 

Tills  appendix  does  not  depend  on  the  preceding  material. 

Let  (f^)  be  a  sequence  of  functions  with  vsilues  in  the  extended  real 
Jine,  We  give  suffj.cient  conditions  that 

(66)  lim  min  f  (y)  =  min  lim  f  (y) 

n-*oo  yeY  yeY  n-*» 

Although  apparently  not  in  the  literature,  the  following  result  is  prob¬ 
ably  well  known  and  part  of  dynamic  prograjmmlng  folklore : 

LEMMA.  If 

(i)  Y  is  compact 

(ii)  f  (y)  is  continuous  in  y  , 

n 

(iii)  f^  f  ,  uniformly  over  Y  , 
then  (66)  holds, 

PROOF.  The  first  two  conditions  are  included  only  to  ensure  that  all  mini¬ 
ma  in  (66)  exist,  since  a  continuous  f'anction  over  a  compact  domain  achieves 
its  Infimum;  f  is  continuous,  since  it  is  the  uniform  limit  of  continuous 
functions. 

-)(-  #  a- 

Let  y  gY  be  such  that  f(y  )  «  min  f(y)  and  y  gY  be  such  that 

y€Y  " 

f  (y  *)  =  min  f  (y),  n  =  1,2,....  It  suffices  to  show  that 
yeY 

(67)  |f„{y„*)  -  f(y*)  1-0(1)  . 
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Suppose  that  (67 )  does  not  hold.  Then  there  exists  e  >  0  such  that 

(68)  I  )  -  f  (y  )  I  >  €  for  infinitely  many  n  . 

However,  by  (iii),  there  exists  n'  such  that 

t 

(69)  I  -f(y)  1<  €  ,  ycY,  • 

#  # 

Whe  Jier  f^(y^  )  ^  f  (y  )  or  vice  versa,  it  follows  that 

(70)  I  I  ^  max  (  I  I  >  I  I  ) 

<  €  ,  vn  >  n'  , 

contreidlcting  (68).  |1 

We  have  not  yet  specified  a  topology  for  Y  .  In  our  case,  Y  was 
a  compactified  half  line  [€,»]  with  the  usual  topology. 
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Appendix  IV 


Outline  of  Replacement  Policy  Computation 

To  simplify  matters,  ve  assume  that  the  grid  ^  Is  given.  See 
Chapter  VI  for  further  details.  An  outline  of  the  replacement  policy 
computation  Is  given  by  the  following  algorithm: 


1. 

2. 


Compute  Rjj^(b,c)  for  all 


N  -  2  . 


3. 


k. 


Compute  R^jj(b,c)  for  all  (b,c)rb  using  the  previously 
computed  values  of  Rj^  [Note  that 


If  max 


(b,c)4l 


.l(b,c).Ri„(b,c) 


<  € 


go  to  7;  otherwise  go  to  5  . 


5.  N  -  N+1  . 

6.  Go  to  5  . 

7.  Exit  with  a  table  of  Rj^j^(b,c)  and  a^j^(b,c)  values. 
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